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Abstract 



^ ^ The problem of secure lossy source-channel wiretapping with arbitrarily correlated side informations at both 

O ' receivers is investigated. This scenario consists of an encoder (referred to as Alice) that wishes to compress a 

source and send it through a noisy channel to a legitimate receiver (referred to as Bob). In this context, Alice 
. must simultaneously satisfy the desired requirements on the distortion level at Bob, and the equivocation rate at 

1/^ . the eavesdropper (referred to as Eve). This setting can be seen as a generalization of the conventional problems of 

in 
in 

, the rate-distortion-equivocation region for the case of arbitrary channels and side informations are derived. In some 

1/^ ^ special cases of interest, it is shown that separation holds. By means of an appropriate coding, the presence of any 

statistical difference among the side informations, the channel noises, and the distortion at Bob can be fully exploited 
in terms of secrecy. 



I. Introduction 



5-H ■ Consider a system composed of three nodes (or sensors) where each one is measuring an analogue source (or 

random field) as a function of time. In order to make reliable decisions, one of these sensors (referred to as Bob) 
can be helped by another one (referred to as Alice), which will transmit some compressed version of its own 
measurement through a noisy wireless channel. The third sensor (referred to as Eve) can listen to the wireless 
medium, and capture some information during the communication. Considering that Eve is not to be trusted (she 
is an eavesdropper), Alice wishes to leak the least possible amount of information about its source. 

The above scenario involves most of the major information-theoretic issues on (secure) source and channel coding. 
In fact, the information-theoretic notion of secrecy was first introduced by Shannon in fTl, where security is measured 
through the equivocation rate, i.e., the remaining uncertainty about the message, at Eve. In terms of source coding. 

The work of J. Villard is supported by DGA (French Armament Procurement Agency). This research is partially supported by the FP7 
Network of Excellence in Wireless COMmunications NEWCOM++. 



May 2011 



DRAFT 



TO BE PRESENTED AT ISIT 2011 



2 



Slepian and Wolf and Wyner and Ziv O introduced the problem of source coding with side information at the 
decoder The corresponding secure scenarios i.e., involving an eavesdropper with its own side information, have 
been recently studied in JU-lHl. Secure source coding scenarios involving a secure rate-limited channel between 
Alice and Bob, which allows the use of secret keys, have also been studied in various works ll9l- lfT2l . On the other 
hand, extensive research has been done during the recent years on secure communications over noisy channels. The 
wiretap channel was introduced by Wyner [13], who showed that it is possible to send information with perfect 
secrecy as long as the channel of Bob is less noisy than the channel of Eve. Csiszar and K'orner lfT4l extend 
this result to the setting of general broadcast channels with arbitrary equivocation rate (allowing also a common 
message to both receivers). Several extensions of the wiretap channel have since been done (cf. ifTOl . ifTSl - lfTTl 
and references therein). Whereas, secure lossy source-channel coding problems have received fewer attention. In a 
recent work ifTSl . Merhav considered such a setting by assuming that Eve has a degraded channel with degraded 
side information with respect to Bob, and that a secret key can be shared between Alice and Bob. 

In this paper, we investigate the general problem of secure lossy source-channel wiretapping, with arbitrarily 
correlated side informations as depicted in Fig. [1] The main goal is to understand how Alice can take advantage of 
the presence of statistical differences among the side informations and the channel noises to reveal the minimum 
amount of information to Eve, and satisfy the required distortion level at Bob. It should be emphasized that the central 
difficulty of this problem lies in the evaluation of the equivocation at Eve. We derive single-letter characterizations 
of inner and outer bounds on the general rate-distortion-equivocation region (in Section |ll]i. Section |lll] provides 
special cases for which separation holds. The sketches of the proofs are relegated to Sections |IV] and |V] Finally, 
Section IVTl presents discussions and an application example to binary sources. 

Notations 

For any sequence (a;i)ieN*, notation xJJ stands for the collection {xk,Xk+i, ■ . ■ ,a;„). a;" is simply denoted by 
x". Entropy is denoted by H{-), and mutual information by /(■; ■). Let X, Y and Z be three random variables 
on some alphabets with probability distribution p. If p{x\y, z) = p{x\y) for each x, y, z, then they form a Markov 
chain, which is denoted hy X ^ Z. The set of nonnegative real numbers is denoted by R+. For each a: £ M, 
notation stands for max(0;a;). 

II. Problem Definition and Main Results 

A. Problem Definition 

In this section, we give a more rigorous formulation of the context depicted in Fig. [T] Let A, B, £, X, y, and Z 
be six finite sets. Alice, Bob, and Eve observe the sequences of random variables (Ai)igN*, (Si)ieN*, and {Ei)il=^^, 
respectively, which take values on A, B, and £, resp. For each i e N*, the random variables Ai, Bi, and Ei are 
distributed according to the joint distribution p(a, h,e) on A x B x £. Moreover, they are independent across time 
i. Alice can also communicate with Bob and Eve through a discrete memoryless channel with input X on X, and 
outputs Y, Z on y, Z, respectively. This channel is defined by its transition probability P{Y Z\X). 
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A" : < D 



^H{A"\E"Z"') > A 



Figure 1: Secure lossy source-channel wiretapping in the presence of side information at the receivers. 



Let d : Ax A ^ [0 ; dmax] be a finite distortion measure i.e., such that < dmax < oo. We also denote by d the 
component-wise mean distortion on A^ x i.e., for each a", 6" G A''\ d{a^\ &") = ^ ^7=i ^{'^ii ^O- 
Definition 1: An (n,TO)-code for source-channel coding in this setup is defined by 

> A (stochastic) encoding function at Alice F : A"' X™, defined by some transition probability Px"^\A^{'\')^ 
m A decoding function at Bob g : >"™ x B^' ^ A". 

The rate of such a code is defined as quantity m/n (channel uses per source symbol). 

Definition 2: A tuple {k,D,A) e R'^ is said to be achievable if, for any e > 0, there exists an (n,m)-code 
iF,g) s.t.: 

— <k + e , 
n 

E[d(^",(?(y",B"))] <£> + £, 

n 

when the input of the channel X'" is the output of the encoder F{A'^). The set of all achievable tuples is denoted 
by TZ* and is referred to as the rate-distortion-equivocation region. 

B. Main Results 

The following theorem gives an inner bound on TZ* i.e., it defines region TZ^a C TZ* . The proof is outlined in 
Section |IVl 

Theorem 1 (Inner Bound): The set of all tuples (fc, D, A) in K'^ such that there exist random variables U, V, Q, 
T on some finite sets U, V, Q, T, respectively, with joint distribution p{uvqtabexyz) — p{u\v)p{v\a)p{abe)p{q\t) 
p{t\x)p{xyz), and a function A -.V x B ^ A, verifying the following inequalities, is achievable: 

I{U;A\B) < kI{Q;Y) , 
IiV;A\B) < kI{T-Y) , 

D > E[d(A, A{V, B))] , 

A < H(A\UE) - I{V; A\UB) - k{l{T; Y\Q) - I{T; Z\Q)'^ 

The first two inequalities in Theorem [T] correspond to sufficient conditions for the transmission of two source 
layers U, V in channel variables Q, T, resp. The first layer ([/ Q) can be seen as a common message which is 



May 2011 



DRAFT 



TO BE PRESENTED AT ISIT 2011 



A" 



Source 


r 


Channel 


encoder 




encoder 



A" 



Source 




Channel 




encoder 




encoder 





Figure 2: Traditional ("informational") separation. Figure 3: Proposed system ("operational" separation). 



considered to be known at Eve, as shown by the term H{A\UE) in the equivocation. The second layer {V T) 
forms a private message which is (partially) protected by adding an independent random noise [1741 . ifTTl . The term 
inside the brackets in the fourth inequality corresponds to the information that Eve can still obtain on this protected 
layer. 

The following theorem gives an outer bound on TL* i.e., it defines region TZom 3 Tl* ■ The proof is outlined in 
Section |V] 

Theorem 2 (Outer Bound): For each achievable tuple (fc, Z?, A), there exist random variables U, V, Q, T on 
some finite sets U, V, Q, T, respectively, and a function A -.V xB ^ A, such that p{uvqtabexyz) — p{uv\a)p{abe) 
p{q\t)p{t\x)p{xyz), and 

I{V;A\B) < kI{T;Y) , 

D > E[d{A, A{V, B))] , 

A< H{A\UE) - I{V;A\B) - I{U;A\B) -k(^I{T;Y\Q) - I{T;Z\Q)j ^. 

Notice that the inner and outer bounds do not meet in general. In Section |llll we provide several cases where 
7?,in is optimal. In fact, there are two main differences between TZin and 7?.out: 

• The first inequality of Theorem [T] which is needed in our scheme to characterize the equivocation at Eve, may 
not be optimal for the general case, 

• The Markov chain U -e- V -e- A -e- {B, E) is assumed in Theorem [T| while only ([/, V) -e- A -b- [B, E) is 
proved for arbitrary codes in Theorem 

C. Coding Scheme Based on "Operational " Separation 

In traditional separated schemes, two stand-alone components successively perform source and channel coding, 
as depicted in Fig. |2] However the proposed scheme (which achieves region TZin) does not satisfy this separation 
principle: The source encoder outputs two layers (as in H) which are further encoded by using the channel code 
for a broadcast channel with a confidential message |fT4l. This results in two independent (but not stand-alone) 
source and channel components leading to statistically independent source and channel variables (as in ifTSi for 
Slepian-Wolf coding over broadcast channels) i.e., "operational" separation holds (see Fig. |3). As a matter of fact, 
the first inequality of Theorem [T] i.e., I{U\A\B) < kI{Q;Y), prevents from separately choosing variables U and 
Q which would maximize the equivocation rate at Eve. 
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III. Special Cases of Interest 



In this section, we characterize the optimaUty of the inner bound TZm for some special cases. 

A. Bob Has Less Noisy Side Information 

Definition 3: Random variable B is less noisy than E w.r.t. A, if I{U; B) > I{U ; E) for each rv. U s.t. 
U -e- A -e- (B, E) form a Markov chain. This relation is denoted by _B E. 

Proposition 1: If B 4 E, then region TZ* reduces to the set of all tuples {k, D, A) G M']_ such that there 
exist random variables V, Q, T on some finite sets V, Q, T, respectively, with joint distribution p{vqtabexyz) = 
p{v\a)p{abe)p{q\t)p{t\x)p{xyz), and a function A : V x B ^ A, verifying the following inequalities: 



Remark 1: In this case, the optimal coding reduces to a Wyner-Ziv source encoder ||3] followed by a classical 
wiretap channel encoder ||T41 . IfTTl . and hence the conventional separation principle holds (Fig. |2]l. 

Proof: The above region is achievable by setting variable [/ to a constant value in Theorem [T] On the other 
hand, the third inequality of Theorem |2] writes: 



Since B E, and U ^ A ^ {B,E) form a Markov chain, I{A;B\U) - I{A;E\U) < I{A;B) - I{A;E). 
Moreover H{A\UE) < H{A\E). In this case, the outer bound TZom is thus included in (and consequently equal 



If the informations at Eve (both side information, and channel output) are degraded versions of Bob's ones i.e., 
if both Markov chains A -e- B -e- E, and X -b- Y -e- Z hold, then Proposition [T] reduces to the results in |15|. 
In this case, variable Q is set to a constant value, and T = X. 

B. Eve Has Less Noisy Channel Output 

Proposition 2: If Z Y, then region TZ* reduces to the set of all tuples {k,D,A) e R'^ such that there 
exist random variables U, V on some finite sets U, V, respectively, with joint distribution p{uvabexyz) — p{u\v) 
p{v\a)p{abe)p{xyz), and a function A : V x B ^ A, verifying the following inequalities: 



I{V;A\B) < kI{T;Y) , 



D > E[d{A, A{V, B))] , 




+ 



A < H{A\UE) 

A < H{A\VB) + I{A; B\U) - I{A; E\U) - k[l{T; Y\Q) - I{T; Z|Q)) . 



to) 7e,n. 



I{V;A\B) < kI{X;Y) , 



D > E[d{A,A(V,B))] 



A < H{A\VB) + I{A; B\U) - I{A] E\U) . 
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Remark 2: In this case, the optimal scheme reduces to a secure source encoder ||8] followed by a conventional 
channel encoder, and hence separation principle holds (Fig. |2|i. 

Proof: The above region is achievable by setting Q = T = X in Theorem[T] However, a new proof is needed to 
obtain the converse part of Proposition |2] Here, auxiliary variables are defined as follows, for each i e {!,..., n}, 
and each j £ {1, . . . , m}: 

V, = {A'-\,B'-\B^_^^,E'-\Y"') , 

Now, both Ui -e- -e- A.^ -e- {Bi,E,i), and Qj ~e- Tj -e- Xj -e- (Yj-, Zj) form Markov chains. Following the 
arguments given at Section |V] we can define new auxiliary variables verifying the above Markov chains and the 
following inequalities: 

I{V;A\B)<kIiT;Y) , 

D > E[d{A, A{V, B))] , 

A < H{A\UE) - I{V- A\UB) + k(l{T- Y\Q) - I{T; Z\Q) 



Since Z hx Y, and Q ^ T ^ X ^ {Y, Z) form a Markov chain, /(T; Y\Q) - I{T; Z\Q) < 0. Noting that 
I{T; Y) < I(X; Y), this concludes the proof. ■ 
Defining the transmitted rate as i? = kI{X;Y), Proposition |2] provides the rate-distortion-equivocation region 
in the secure source coding setup [S] Theorem 1]. 

IV. Sketch of Proof of Theorem[T](Inner Bound) 

The proof is based on the use of a secure source coding scheme [8|, and a channel coding scheme for wiretap 
channel lfT4l . ifTTl . Full details are omitted due to the lack of space and will be provided in an extended version of 
this paper. 

Source Encoder: The source encoder is formed of two layers corresponding to variables U, V, with respective 
rates Ri, i?2- Random binning a la Wyner-Ziv |3| is performed prior to transmission. The next constraints ensure 
that Bob can decode {U, V) from bin indices (ri, with an arbitrarily small error probability: 

Ri > I{U;A\B) , 
R2 > I{V;A\UB) . 
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Bits Recombination: Bin indices (ri, are mapped to indices Tc and r^, with respective rates Rc, Rp, through 
a one-to-one mapping, such that ri = M'{rc) for some mapping M' . This requires the following constraints: 

i?l + i?2 = fic + -Rp J 

Channel Encoder: The channel encoder is composed of two layers corresponding to variables Q, X, transmitting 
messages r^, Vp, respectively. Following lfT4l . IfTTl . an independent random noise ry, with rate Rf s.t. i?j < 
kI{X; Z\Q), is also transmitted with message r^. The following constraints ensure that Bob can decode r^, {rp^rj) 
from his channel output Y with an arbitrarily small probability of error: 

R, < kI{Q;Y) , 
Rp + Rf < kI{X;Y\Q) . 

Distortion at Bob: Provided the above constraints are verified. Bob can decode V with an arbitrarily small 
probability of error, and compute an estimate A of A with mean distortion E[(i(yl, A(y, _B))]. 

Equivocation Rate at Eve: After some algebraic manipulations, it can be proved that the proposed scheme 
achieves any equivocation rate verifying the following inequality: 

A < H{A\UE) - R2 + Rp + Rf - kI{X; Z\Q) . 

The proof (which is omitted here due to the lack of space) follows the arguments of both (W Section IV- A], and IfTTl 
Section 2.3], and relies on relation ri = M'(rc). 

End of Proof: Putting all inequalities together, using Fourier-Motzkin elimination, and prefixing an arbitrary 
DMC P{X\T) to the DMC P{Y, Z\X) prove Theorem [I] ■ 

V. Sketch of Proof of Theorem[2](Outer Bound) 

Due to the lack of space, we only provide some of the basic ideas underlying the proof of Theorem |2] Details 
will be provided in an extended version of this paper 

For each i G {1, . . . , n} (resp. each j e {!,..., m}), define the source (resp. channel) auxihary random variables 
Ui, Vt (resp. Qj, Tj) as 

V, = {A'-^,B'-\,Bl'_^^,E''\Y"') , 
Note that {Ui,Vt) ^ A^ [Bi, Ei), and Qj ^ Tj Xj iYj,Zj) form Markov chains. 
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Rate: Using the chain rule for conditional mutual information, the Markov chain (yl^, y™)-e' (A* ^,_B")-e-i?' ^, 
and the fact that random variables Ai, Bi, and Ei are independent across time, we can prove that /(A"; = 

From the chain rule, and the non-negativity of mutual information, we can also prove the following upper bound: 

The above equations yield 

n m 

J2l{A;V.m<J2liT,;Y,) . 

i=l j=l 

Distortion at Bob: Bob reconstructs g(Y"^, S"). The i-th coordinate of this estimate is gi(Y"^ , B'^^ , Bi, Bf_^^) = 
Ai{Vi,Bi). The component-wise mean distortion at Bob thus writes: 

1 " 

E[d(A",5(y",i?"))] = -J2E[d{A,MV,,B,)) . 

i—1 

Equivocation Rate at Eve: From the chain rule for conditional entropy, and the Markov chain Ai-e-{A2_^i^ ^ Z^) 
-e~ (_SJYi, ^J+i), we can prove the following upper bound on the equivocation at Eve: 

n 

77(A"|£;"z™) <^i/(A,|c/,i;o . 

Using the Markov chain -e- -e- Z™, we expand the equivocation at Eve as follows: 

^ v ' ^ V ' 

Ae A, 

Following m Section V], W\ Section 2.4], we can prove that = X^Jli I{Tj;Yj\Qj) - I{Tj; Zj\Qj), and 
following m Section IV-B], A, = YJU H{A,\V^Bi) + I{A,; B,\U,) - I{Af, E,\U^). 

End of Proof: Following the usual technique, we define independent random variables K, and J, uniformly 
distributed over the sets {l,...,n}, and {l,...,m}, respectively. We also define random variables A = Ak, 
B = Bk, E ^ Ek, U = [K, Uk), V = {K, Vk), X ^ Xj,Y ^Yj, Z = Z,j, Q ^ (J, Qj), and T = (J, Tj). 
{U,V) -e- A {B,E) and Q T X {Y, Z) still form Markov chains. Using these definitions, we 
prove the three inequalities of Theorem |2] Since they only involve marginal distributions of auxiliary variables, 
w.r.t. corresponding source/channel variables i.e., p{uv\a) and p{qt\x), we can define new auxiliary variables U, 
V, Q, and T, with identical marginal distributions, such that the (global) joint distribution writes p{uvqtabexyz) = 
p{uv\a)p{abe)p{q\t)p{t\x)p{xyz) i.e., source and channel variables are independent. ■ 

VI. Application Example and Discussion 

Consider the source model depicted in Fig. |4] where the source is binary and the side information at Bob, resp. 
Eve, is the output of a binary erasure channel (BEC) with erasure probability /3 G [0, 1], resp. a binary symmetric 
channel (BSC) with crossover probability e G [0, 1/2], with input A. The communication channel is similar to the 
one of [13|: It consists of a noiseless channel from Alice to Bob, and a BSC with crossover probability C G [0, 1/2], 
from Alice to Eve. 
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1-/3 1-e 
Figure 4: Binary erasure/binary symmetric side informations. 



This model is of interest since neither Bob nor Eve can always be a lessnoisy decoder for all values of e). 
Let /i2 denotes the binary entropy function given by h2{x) = — 2;log2(a;) — (1 — a;) log2(l — x). According to the 
values of the parameters e), it can be shown by means of standard manipulations |fT9l that the side informations 
satisfy the properties summarized in Fig. |5] 

2e 4e(l-e) /12(e) /? 



A^B^E B>aE I{A;B)>I{A;E) 

Figure 5: Relative properties of the side informations as a function of (/?, e). 

From now on, let the distortion level at Bob be zero i.e., he performs lossless reconstruction, and assume for 
simplicity that the source is uniformly distributed i.e., {A = 0} = Pr{A = 1} = 1/2. We focus on rate fc = 1 
channel use per source symbol. Under these assumptions, the inner bound of Theorem [T] is maximized by choosing 
V = A and a uniformly distributed binary auxiUary random variable U (resp. Q), produced as the output of a 
BSC with crossover probability u e [0, 1/2] (resp. q G [0, 1/2]), and input A (resp. X), as stated by the following 
proposition (which proof is omitted due to the lack of space). 

Proposition 3: In the case considered in this section, region 7?,in reduces to the set of all tuples {k — 1,D — 0,A) 
such that there exist u,q <E [0, 1/2] satisfying 

Pil - h2{u)) < 1 - h2{q) , 

A<h2 (e) + h2iu)-h2ie*u)- [/3/i2 (u) - (/i2 (C) + (g) - /l2 (C * <?)) ] ^ , 

where a-kb ^ a(l — fe) + (1 — a)b for each a,b G [0, 1]. 

Notice that if (3 < 4e(l — e) then B E, and hence Proposition [T] holds i.e., the above inner bound is optimal. 

Counterexample for the optimality of Theorem Q] 

Let now assume that Bob does not have any side information i.e., /3 = 1, and let e = = 0.1 so that A-e- E ^ B 
form a Markov chain, and neither Proposition [T] nor Proposition |2] applies. This setting provides a counterexample 
for the general optimality of the inner bound in Theorem [T] Numerical optimization over u and q in Proposition |3] 
indicates that the proposed scheme achieves an equivocation rate A — 0.056, while a naive analogue scheme 
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consisting of directly plugging the source on the channel achieves A = 0.258. Furthermore, the latter concides with 
the outer bound of Theorem |2] 

The above example shows that a naive joint source-channel scheme may achieve better performance in some 
cases. At first look, this is not surprising since it is well-known that joint source-channel coding/decoding is a must 
for broadcast channels without secrecy constraints ll20l . ifTSl . However, the secure setting is rather different because 
Alice only wants to help one receiver (Bob), while she wants to blur the other one (Eve). Therefore, the intuition 
indicates that the optimal strategy would be the opposite i.e., separation between source and channel encoders, as 
in Propositions [T] and |2] 
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